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AMENDMENTS TO THE DRAWINGS 

The attached sheets of drawings include changes to Figs. 3, 5, 13 and 14. These sheets 
replace the original sheets including Figs. 3, 5, 13 and 14. In Fig. 3 5 the spelling of SEARCH at 
S3 is corrected; in Fig. 5, the spelling of ORIENTATION at SI 8 is corrected, in Fig. 13, the 
spelling of SEARCH at S102 and S107 is corrected; and in Fig. 14, the spelling of SEARCH at 
S202 and S208 is corrected. 
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REMARKS/ARGUMENTS 

Claims 1 5 2, 6, 9 and 12 are now active in this application. Claims 3-5 , 7, 8, 10, 11 and 
13 are withdrawn as being directed to non-elected species. 

REJECTION OF CLAIMS UNDER 35 U.S.C. $ 103 

I. Claims 1, 2, 6 and 12 are rejected under 35 U.S.C. § 103(a) as being unpatentable over 
Zhang et al. (USPN 6,671,391). 

Claim 9 is rejected under 35 U.S.C. § 103(a) as being unpatentable over Zhang et al. 
(USPN 6,671,391) in view of Hagiwara et al. (USPN 6,816,61 1). 

II. Zhang et al. (USPN 6,671,391) was filed in the U.S. on May 26, 2000. However, the 
present invention was conceived in Japan prior to May 26, 2000, as evinced by the attached 
DECLARATION UNDER 37 CFR §1.131 by Applicant/Inventor Kyoko Nakamura. 

In view of this Declaration, and the evidence presented therein, withdrawal of the 
rejection of claims 1, 2, 6 and 12 under 35 U.S.C. § 103(a) as being unpatentable over Zhang et 
al. (USPN 6,671,391), and of claim 9 under 35 U.S.C. § 103(a) as being unpatentable over 
Zhang et al. (USPN 6,671,391) in view of Hagiwara et al. is respectfully solicited, as the present 
invention was conceived in Japan prior to May 26, 2000, the date Zhang et al. (USPN 
6,671,391) was filed in the U.S. 

CONCLUSION 

Accordingly, it is urged that the application, as now amended, is in condition for 
allowance, an indication of which is respectfully solicited. If there are any outstanding issues 
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that might be resolved by an interview or an Examiner's amendment, Examiner is requested to 
call Applicants' attorney at the telephone number shown below. 

To the extent necessary, a petition for an extension of time under 37 C.F.R. 1.136 is 
hereby made. Please charge any shortage in fees due in connection with the filing of this paper, 
including extension of time fees, to Deposit Account 500417 and please credit any excess fees to 
such deposit account. 



Respectfully submitted, 



McDERMOTT WILL & EMERY LLP 




600 13 th Street, N.W. 
Washington, DC 20005-3096 
Phone: 202.756.8000 EJWxac 
Facsimile: 202.756.8087 
Date: November 27, 2005 



Please recognize our Customer No. 20277 
as our correspondence address. 
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Assignee 



Osaka Kokusai Building, 3-13, Azuchimachi 2-chome, Chuo-ku, Osaka 
Minolta Camera Co., Ltd 

Representative Director & President, YoshikatU Ohta 



(Seal) 



Notification of Invention, Device and Creation, and Assignment 



Date: March 15. 2000 



1 . Title of article relating to invention, device and design 
Image selector 


2. This notification concerns the above invention, device or design and has been prepared as an assignment to certify that all rights for acquiring the 
patent, utility model and design attributing to them have been assigned to Minolta Camera, including the rights for foreign application and divisional 
application. 


3.Creator of invention and device(assignor) 


Name of assignor 


Seal (Note) 


Address of assignor 


Employee 
code 


Department and section 
names 


Contribution ratio 
(blank: equability) 


Kyoko Nakamura 




do Minolta Camera Co., Ltd, Osaka Kokusai Building, 
3-13, Azuchimachi 2-chome, Chuo-ku, Osaka 


16811 


Electric Information 
Engineering Division 

































































Space for inventor and creator (assignor) 


4. Special notes written on the rear (descriptions on restoration compensation) 
have been read and accepted prior to this notification. 


5. Special notes 



6. Relevance to research & 
development 
Research & development 
theme code 
Product code 



7. Type and urgency of 
application 



Product group code 



0 
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9 





































Functional classification 1 0 

a □ Adopted Predetermined To be determined 

Adopted Predetermined To be determined 

I a D Adopted n Predetermined Q To be determined 

Adopted Predetermined D To be determined 



liJ Patent U Utility model 

(LJ Division Domestic priority (basic application sequence number: ) 

D Design (O Relation (principal design sequence number: ) D Trade secret i 

Urgency of application EI Regular □ Urgent (Deadline: ) 

Reason: 



.year) 



8. Presence of special reason 
(contract, etc.) 
Necessity of joint application 
Necessity of follow-up for 
domestic priority 
Others 



Uno U Yes (Company name: ) 

E3 No □ Yes [Procedure ( □ Self □ Partner) Expense (□ 50-50 basis □ Self □ Partner)] 
H No d Yes (Expected date for filing notification of invention for domestic priority: 



Section code | 2 | 2 | 3 | 0 



9. Decision result at application in Japan after consultation with Intellectual Property department 
Application in Japan: GO Yes L~H No 



□ Held □ To be publicly known 

Request for examination: CD Yes 

Held at application 



Foreign application: H Yes D No 

Application country Q USA 

□ EPC [Designated country: _ 

□ Others [ ~ 



Approved by 



Rank: 



| A<B)C *D 



Special note: (Reason for difference from automatic determination) 
2-2-B 



Pattern: | 1 • II * 111 



10. Li) Approval of determination at domestic application 

D Approval of determination at domestic application through the following amendments 

Domestic application (Japan) 

Request for examination in Japan 

Foreign application 



Approved by 



Space for Intellectual Property Department 



Approved by Intellectual 
Property Department manager 



Approved by Intellectual 
Property Section manager 



Liaison number 


Liaison 


Specialist 


Agent 




Sequence number 


TL3619 


Kobayashi 






Fukami 


P23522 



Chisen-l 



Description of Invention and Device 

1. Name of Inventor and Creator (Underline the name.) 

Kyoko Nakamura 

2. Title of Invention and Device 

Image selector 

3. Summarize the advantage of invention (or device). 

0 A face facing a designated direction is detected during input image scanning. Once it is detected, 

its image is selected and captured. 
0 A face facing a designated direction is repeatedly detected and counted. When the number of 

detected faces is equal to, greater than or less than the predetermined count, its image is 

selected and captured. 

4. Describe the purpose of this invention or the background, motivation, chance, design 
task, research task, etc. for this invention. 

When taking a group picture, it is difficult to click the shutter at the timing when multiple persons 
face the front. In particular, when objects are grade school children or younger infants, it is difficult 
to shoot a group picture of them because they don't face the front meekly regardless of commands. 
In addition, when taking a picture of babies, it frequently fails to click the shutter at the moment 
when their eyes catch the camera (lens) because they are restless and cannot be easily caught within 
an angle of field or it is difficult to click the shutter at the timing when they look natural because 
they are conscious of the approaching camera. When taking advantage of this invention, the 
photographer need only to select the face selection mode, designate the direction of face, and then 
monitor the timing when the object is caught in a balanced manner with the finder. When the object 
faces the designed direction, the shutter is clicked automatically. Owing to this automatic click, the 
natural expression of the object (baby or infant) before being conscious of the approaching camera 
can be shoot or even the photographer not having enough shooting experience can shoot even when 
the object is restless and cannot be caught within an angle of field without difficulty. 

5. Describe the patent gazette, document and product related to prior arts. In addition, 
summarize the prior art and differences from this invention. 



Kokai (unexamined patent publication) No. 1997-281605, Image Recorder, Fuji Photo Film Co., 
Ltd. 

The number of persons photographed is counted and as many prints as persons are produced 
automatically. 

Kokai (unexamined patent publication) No. 11-196398, Image Processor, Sony Corporation 

A face direction can be easily measured. The angle of an object is calculated according to the 
degree of correlation between a fullface image previously registered by multiple cameras and 
each of the images captured by the respective cameras. 

The above patents are typical of prior art. However, these arts do not take advantage of face 
image direction and perform nothing according to comparison result (equal to or larger than 
the number of images counted). 

6. Describe a working example of this invention. 

The following shows a working example of this invention. 

Example 1 : This invention applies to a digital/silver salt camera. 

Figure 1 shows a regular camera provided with a function switch button 5, a direction 
specification button 6 and a confirmation screen 4. The function switch button 5 
toggles the camera functions such as flash usage and self-timer usage. This function 
switch button is used to designate the face photography mode. In this face 
photography mode, either the count preference mode or the time preference mode can 
be designated. When the count preference mode is designated, the face selection 
mode remains effective until the number of facial portraits facing the designated 
direction reaches the designated count. In the time preference mode, shooting takes 
place each time a face with a designated angle is found within a specified period of 
time. When no face with a designated angle is found within the period, no shooting 
takes place and the face photography mode terminates. The direction specification 
button 6 is used to designate an angle of the face to be photographed. On the 
confirmation screen 4, various camera settings (for example, number of remaining 
sheets for photography, time and current photography mode) can be confirmed in 
addition to confirmation in the face photography mode. This confirmation screen can 
be displayed on the LCD provided to the rear cover of the camera or within the finder. 
The photographer can use the function switch button 5 to select the face selection 
mode and decide a direction of the face to be photographed. After making this setting, 
the photographer clicks the shutter to start shooting. 

The following describes an operation flowchart in the face photography mode. In 
steps #001 and #002, the mode is selected and a face direction is designated. In step 
#003, a face with a designated direction is retrieved among input images. Typical 



algorithms for searching for faces with a designated direction are shown in technical 
papers such as H.A. Rowley, "Neural Network-Based Learning for View-Based 
Human Face Detection", CMU-CS-99-117, 199 and H. Schneiderman, T. Kanade, 
"Probabilistic Modeling of Local Appearance and Spatial Relationships for Object 
Recognition", CVPR98. When no target face is found, step #003 is repeated to find 
the face. When it found, shooting (image capture) in step #005 takes place. When 
photography does not terminate because of the specified time or the specified number, 
processing returns to step #003 and face selection and photography takes place. 
Otherwise, processing proceeds to step 7 and terminates. 

When taking a picture (portrait) of baby, it sometimes fails to click the shutter at the 
moment when their eyes catch the camera (lens) because they are restless and cannot be 
easily caught within an angle of field or it is difficult to click the shutter at the timing 
when they look natural because they are conscious of the approaching camera. When 
taking advantage of this invention, the photographer need only to select the face 
selection mode, designate the direction of face, and then monitor the timing when the 
object is caught in a balanced manner via the finder. The shutter is clicked only when 
the object faces the designed direction. Owing to this shutter click it is possible to take a 
picture of babe at the moment when the baby turns to the direction of the camera 
without knowing the presence of camera or even when he or she is restless. 

Example 2: A head count setup function is added to example 1 

A camera is provided with a function switch button 5, a head count button 6 and a 
confirmation screen 4. The function switch button 5 toggles the camera functions 
such as flash usage and self-timer usage. The head count button 6 is used to designate 
the number of objects facing the designated direction (for example, front) during 
photography. On the confirmation screen 4, various camera settings (for example, 
number of remaining sheets for photography, time and current photography mode) 
can be displayed. This confirmation screen 4 can be displayed on the rear cover of the 
camera or within the finder. 

The photographer can use the function switch button 5 to select the face selection 
mode and decide a direction of the face to be photographed. In the face photography 
mode, either the count preference mode or time preference mode can be designated. 
When the count preference mode is designated, the face selection mode remains 
effective until a face with a designated direction is photographed a specified number 
of times. When the time preference mode is designated (selected), photography takes 
place each time a face with a designated direction is found within a specified period 
of time. When the target face can be found, the face photography mode terminates 
without photography. The head count button 6 is used to define the head count facing 



the designated direction. After making the setting, the photographer clicks the shutter 
2 to terminate photography. 

The following describes an operation flowchart of the camera in the face photography 
mode. In steps #001 and #002, the mode and head count are decided. In step #003, a 
face with a designated direction is retrieved among input images. 
In step #004, the number of face with the designated direction which is detected in 
step #003 is counted. When the number of counted faces is not equal to the 
designated head count, processing returns to step #003 and retrieving the target face 
continues. When it is equal to the designated head count, processing moves to step 
#006 from step #005. In step #007, photography takes place to capture the image. 
When using a camera with this function, the photographer can take advantage of the 
effect equivalent to the self-timer function in taking group pictures during a trip. 
Concretely speaking, the photographer need only designate the front for direction, set 
the head count to be photographed, fix the camera and then join a group of objects so 
that the shutter is clicked automatically when the specified number of objects in the 
group face the front. Unlike the self-timer function, this function eliminates the need 
for quick move to the group. In addition, when taking a group picture with a camera 
held, the photographer cannot easily monitor the expressions of all objects via the 
finder and click the shutter at a good timing. In contrast, when using a camera with 
this function, the photographer need only set the head count and monitor the objects 
to be in a balanced manner via the finder because the shutter is clicked automatically 
to take a group picture of all members facing the designated (correct) direction. 

Though the above examples of this patent are applications to cameras, this patent is 
also available to extract still pictures from motion pictures taken by video recorders. 
In addition, this patent can be used to search for an image in the image database. For 
example, an image having a person facing a particular direction or having three 
persons facing the front can be retrieved using a keyword. 



1 



1 . Camera 4. Confirmation screen 

2. Shutter 5. Function switch button 

3. Finder 6. Direction designation button 
Figure 1 
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Figure 2 Block diagram for example 1 




Means for finding a face 
with a designated direction j 



Means for capturing an j 
image I 



U002 



Means for counting a face 
with a designated direction 




^ Jpos 

Means for selecting an 
image 



Means for setting a head , 
count 



Figure 4 Block diagram for example 2 
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Figure 5 Block diagram for example 2 



Describe the effect of this invention. 

■ Even when the object is a baby who cannot understand the purpose of photography, it is possible to take a picture 
of the baby at the moment when he or she has turned to a direction desirable for the photographer. 

■ Even when there are many objects like a memorial picture or a group picture during a trip, it is possible to take a 
picture of all or many objects facing the same direction under the best condition without failing to catch a 
momentary chance. 



Describe the scope of claims on the effects of this invention. 

■ Means for shooting or means for reading an image 

Means for finding a face with a designated direction: Finds a face with a designated direction among input images. 
Means for selecting an image: Selects and captures a face with a designated direction 

when it is found. 



■ Means for shooting or means for reading an image 
Means for finding a face with a designated direction 

Means for counting a face with a designated direction: Counts a face with a designated direction. 

Means for setting a head count in an image: Designates a head count in an image. 

Means for selecting an image: Selects and captures an image when the number of 

detected objects each having a face with a designated 
direction is equal to, greater than or less than the 
designated head count. 



Embodiment claim 
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Special in-house questions 

Burdens up to completeness of this invention 

Current problems 

Others 
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Proposed number: X002 - 99 - 009 



1st Section Manager. Intellectual Property Department 



Liaison Comment 

Liaison number for this invention: TL 3619 



6th Section, Intellectual 
Property Department, 



Checked by 


Prepared by 







Related invention to be treated by the same agent 



■ Related invention or prior application for technical reference 

TL2817 (Shooting is disabled so long as the corresponding marker stays within a scope of 
shooting.) 



Demand for treatment 

0 Desired agent 

0 Necessity of visit for treatment 
0 Necessity of draft delivery to liaison 
Nothing in particular 



Fukami Patent Office 



□ Required 
■ Required 



■ Not required 
□ Not required 



■ Comment on in-house embodiment and deadline for treatment 

In taking a group picture, the shutter (of the camera) is automatically clicked and a face 
with a particular direction can be automatically selected among image data. 

• Presentation: None 

• Agreement: Peadline for treatment; None 

■ Overview of technology, others 

When a fullface is found among input images, its image is selected (and captured). 



Attached paper (prese nce C omplementary report, absence) 



«Compass and abstract» 

■ Technical explanation When a fullface is detected among input images, its image is selected. 

■ Technical category Classification □ Business machine □ Camera ■ Others 
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Complementary Report 



□ Attached paper present B Attached paper not present 





Liaison num ber: TL 3619 


Developed by: Kyoko Nakamura Theme: Face recognition and synthesisi 


Sequence man her: P 


Liaison: Takao Kobayashi Deadline for application: None 


Summary of invention: Selects an image (takes a picture) at fullface detection during input image scanning. 




Positioning 
Related application 




A s nutter automatically cucks at group pnoxo snooting ana a iace luwctrus* a 
desired direction can be automatically selected among image data. 




TL2817 (Shooting is disabled so long as the corresponding marker stays within a 
scope of shooting.) 




Supplement of claims 




• Claim 1 : It does not matter even for single object. 

• Claim zi 1 ne numoer oi races towaros a aesirea direction can ue counieu 

when there are many objects. 




Supplementary 
explanation 




• Searching for a fullface continues unless the face selection mode is reset. 




Difference ftvm prior art 




• There are already display devices using the principal component analysis 
technology for color space. 




Demand for treatment 
Remarks 




Nothing in particular 
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